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Abstract 

Background: Formulation and evaluation of public health policy commonly employs science-based mathematical 
models. For instance, epidemiological dynamics of TB is dominated, in general, by flow between actively and latently 
infected populations. Thus modelling is central in planning public health intervention. However, models are highly 
uncertain because they are based on observations that are geographically and temporally distinct from the 
population to which they are applied. 

Aims: We aim to demonstrate the advantages of info-gap theory, a non-probabilistic approach to severe uncertainty 
when worst cases cannot be reliably identified and probability distributions are unreliable or unavailable. Info-gap is 
applied here to mathematical modelling of epidemics and analysis of public health decision-making. 

Methods: Applying info-gap robustness analysis to tuberculosis/HIV (TB/HIV) epidemics, we illustrate the critical role 
of incorporating uncertainty in formulating recommendations for interventions. Robustness is assessed as the 
magnitude of uncertainty that can be tolerated by a given intervention. We illustrate the methodology by exploring 
interventions that alter the rates of diagnosis, cure, relapse and HIV infection. 

Results: We demonstrate several policy implications. Equivalence among alternative rates of diagnosis and relapse 
are identified. The impact of initial TB and HIV prevalence on the robustness to uncertainty is quantified. In some 
configurations, increased aggressiveness of intervention improves the predicted outcome but also reduces the 
robustness to uncertainty. Similarly, predicted outcomes may be better at larger target times, but may also be more 
vulnerable to model error. 

Conclusions: The info-gap framework is useful for managing model uncertainty and is attractive when uncertainties 
on model parameters are extreme. When a public health model underlies guidelines, info-gap decision theory 
provides valuable insight into the confidence of achieving agreed-upon goals. 

Keywords: TB management, HIV-AIDS, Public health. Epidemiology, Uncertainty, Robustness, Info-gap 



Background 

Public health policies affect millions of people and deter- 
mine the allocation of health care funds. However, select- 
ing an intervention for a given population at a given time 
is highly uncertain. Data supporting public health deci- 
sions are scarce, of poor quality, not fully generalizable 
and lack appropriate controls [1]. The high uncertainty 
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in infectious disease epidemiology results also from inter- 
dependency among individuals. When prospective studies 
or randomized controlled trials are available, they usually 
represent selected groups with as little variance as pos- 
sible and may not apply to other populations [2]. Such 
lack of generalizability may be more problematic for the 
recommendations developed by international organiza- 
tions. Those guidelines use the best available information 
and expert opinion. Nonetheless the yield, effective- 
ness and cost of the interventions vary significantly due 
to heterogeneity of the populations in which they are 
implemented [1,3]. 
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Science-based mathematical models commonly support 
public health decisions [4-7]. Many models were devel- 
oped to explain or predict the course of an epidemic for 
specific interventions. However, these models are lim- 
ited by the uncertainty of the data and assumptions they 
employ [5,7]. 

Despite severe uncertainty in public health decision- 
making, actions must be timely and cost-effective. Analy- 
sis of uncertainty is central in responsible decision making 
using uncertain data and models. 

Information-gap (info-gap) theory [8] was developed for 
decision making when knowledge gaps are substantial, 
worst cases cannot be reliably identified, and probabil- 
ity distributions are unreliable or unavailable. An info-gap 
model is the disparity between what is known and what 
needs to be known in order to achieve an acceptable 
outcome. The focus is on robustly achieving satisfactory 
outcomes, thus making this technique suitable for pub- 
lic health policy decision making [9]. Info-gap theory has 
been applied in engineering, biological conservation, eco- 
nomics, project management, medicine and homeland 
security (see http://info-gap.com). 

We develop a framework for the practical use of info-gap 
theory in public health for controlling infectious diseases. 
We focus on tuberculosis (TB) in the context of pandemic 
HIV as an example. 

Methods 

Epidemiological background 

The World Health Organization reported 9.4 million inci- 
dent TB cases and 1.7 million TB deaths in 2009 and 
estimated that only 63% of annual incident TB cases 
were detected and reported; of these, 86% were success- 
fully treated [10,11]. Given the disease burden, the United 
Nations Millennium Development Goals include targets 
and indicators related to TB control. The targets include 
decreasing TB incidence by 2015, halving TB prevalence 
and mortality by 2015 (compared with 1990), and diag- 
nosing 70% of new smear-positive cases and curing 85% 
of these cases by 2015. However, despite current efforts, 
many countries will not achieve these targets [10-14]. 

The HIV- AIDS pandemic is the major worldwide chal- 
lenge to TB control [11,13,15,16]. HIV creates a situation 
of serious uncertainty for public health interventions 
based on pre-HIV era models [10,11,13]. This is reflected 
in population distribution, spread, control, and recur- 
rence. Latently and actively infected individuals con- 
tribute differently to spread of disease. It is necessary to 
consider infectivity, rapidity of progression, re-infection, 
individuals with higher susceptibility for infection and 
reinfection resulting from HIV coinfection, etc. in order 
to produce refined models of diagnosis and treatment. 

Many different epidemiological models have been 
used to evaluate treatment strategies. Deterministic 



compartment models are the most common, and we use 
a slightly modified version of the widely used Murray- 
Salomon model [17-19] to describe the evolution of 
TB/HIV epidemics under various scenarios. The details 
of the model appear in Appendix "The Murray-Salomon 
model" section. 

Info-gap theory 

The robustness function is the basic decision-support tool 

in an info-gap analysis. If our dynamic model were accu- 
rate we could evaluate any proposed intervention in terms 
of the outcome of that intervention that is predicted by the 
model. An intervention with low predicted TB prevalence 
is preferred over an intervention with higher predicted 
prevalence. 

The problem is great model uncertainty. This means 
that predicted outcomes are unreliable and it is unrealis- 
tic to prioritize interventions in terms of their predicted 
outcomes. Using the model to find the intervention whose 
predicted outcome is best, is not suited to planning with 
highly uncertain models. 

Model-based predictions are useful, but when decid- 
ing which public health intervention to implement, we 
should also ask: how wrong could the model be, and an 
acceptable outcome is still guaranteed? For any speci- 
fied intervention we ask: what is the largest error in the 
model, up to which all realizations of the model would 
yield acceptable outcomes? Equivalently, what outcomes 
can reliably be anticipated from this intervention, given 
the unknown disparity between the model and reality? 
Answers to these questions lie in the robustness func- 
tion, specified in Appendix "Definition of robustness" 
section. The robustness is dimensionless, and equals the 
greatest fractional error in the model parameters that is 
consistent with a specified outcome requirement. We use 
the robustness function to prioritize the interventions in 
terms of their robustness against uncertainty for achieving 
the required outcome. 

Knight [20] recognized that probability distributions 
are sometimes unknown and that severe uncertainty may 
be non-probabilistic. Wald [21], Ben-Tal and Nemirovski 
[22] and others developed tools for robustly managing 
non-probabilistic uncertainty by minimizing the worst 
outcome on a set of possibilities. Info-gap theory is non- 
probabilistic and handles situations where worst cases are 
unknown. 

We summarize here the main attributes of the info -gap 
robustness function: a plot of robustness-to-uncertainty 
versus required performance. This is the basic info-gap 
tool for prioritizing available options. 

Robustness trades off against performance [23,24] 

More demanding performance requirements are 

less robust against uncertainty than less demanding 
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requirements. This trade off is quantified and expressed 
graphically by the monotonic robustness curve. 

Model predictions have zero robustness against uncertainty 
[25] 

When models are highly uncertain, it is unrealistic to pri- 
oritize one's options based on predicted outcomes of those 
options, because those predictions have no robustness to 
errors in the underlying models. Options must be eval- 
uated in terms of the level of performance that can be 
reliably achieved; this is expressed by robustness. 

Combining the trade off and zeroing properties yields 
realistic prioritization of options. 

Prioritization of options depends on performance 
requirements 

Prioritization of options may change as requirements 

change. This is called "preference reversal" and is 
expressed by the intersection of the robustness curves of 
different options. Preference reversal provides insight to 
anomalous behavior such as the Ellsberg and Allais para- 
doxes in human decision making [8], the equity premium 
puzzle in economics [8], and animal foraging [26]. We will 
show that preference reversal occurs when selecting pub- 
lic health interventions because priorities are time- and 
context-dependent. 

Info-gap models of uncertainty are non-probabilistic 

Info-gap robustness analysis is implementable even 
when probability distributions are unknown, and thus 
is suited to severe uncertainty. In contrast, Monte 
Carlo simulation, Bayesian analysis, or probabilistic risk 
assessment require knowledge of probabilities. Other 
non-probabilistic tools include interval analysis, fuzzy set 
theory [27], possibility theory [28] and Robust Decision 
Making (RDM). A comparison of info-gap and RDM has 
recently been published [29]. 

Info-gap is operationally distinct from the min-max or 
worst-case decision strategy [9] 

Info-gap robustness does not require knowledge of a worst 
case. When even typical scenarios are poorly character- 
ized, it is usually impractical to characterize worst cases, 
which is required by the min-max strategy. Info-gap the- 
ory does require specifying acceptable outcomes. Thus it 
is well suited to policy making, because preferences on 
outcomes are the driving force. 

Info-gap robustness may proxy for the probability of 
satisfying the performance requirement [8,30,3 1] 
A more robust option is often more likely to achieve 
the required outcome. By prioritizing the options using 
info-gap robustness, one maximizes the probability of sat- 
isfying the requirement, without knowing probability dis- 
tributions. The proxy property is central to understanding 



survival in economic [8], biological [26] and other com- 
petitive environments [31]. 

Info-gap implementation 

Info-gap methodology requires three main elements: a 
system model, a performance measure, and a model of 
uncertainty. The system model is a mathematical repre- 
sentation of a system and its influence on the variables 
of interest, for which management aspirations (perfor- 
mance criteria) are set. A performance measure assesses 
value or utility of outcomes. The model of uncertainty is 
a non-probabilistic representation of the degree to which 
the value of parameters, the form of a function, or the 
structure of a model may deviate from nominal estimates. 

The system model in our example is summarized in two 
functions. C{t) is the variation over time of the total num- 
ber of TB cases, untreated and treated, HIV-positive and 
HIV-negative, as a fraction of the initial population. R{t) is 
the total number of relapses, fast and slow, HIV-positive 
and HIV-negative, as a fraction of the initial population. 
(See eqs.(23) and (24) in Appendix "The Murray-Salomon 
model" section.) 

The public health practitioner wishes to control the 
total number of TB cases: the fewer the better. How- 
ever, trying to minimize this prevalence depends on model 
predictions that are highly uncertain. The performance 
requirement is to keep the total fraction of TB cases at 
a specified time, t-m, below a critical value. Cm, eq.(25) in 
Appendix "Performance requirements" section. 

Grassly et al [32] note, in discussing epidemiology of 
HIV/ AIDS, that "not all sources of error are amenable to 
statistical analysis" (p. 137), due to biased, inaccurate or 
unavailable data. The basic idea of info-gap model uncer- 
tainty is that we do not know how wrong our estimates are, 
we have no reliable knowledge of worst cases, and we do 
not know probability distributions for the estimates. The 
info-gap model uncertainty model is a non-probabilistic 
quantification of uncertainties. 

A dominant uncertainty in TB dynamics with HIV 
prevalence is in model parameter values, though HIV 
causes significant uncertainties in model structure. Struc- 
tural uncertainty refers to missing terms in the equations, 
missing equations, or unknown nonlinearities. Structural 
uncertainty is dealt with much less frequently than param- 
eter uncertainty because of technical challenges. We focus 
on parameter uncertainty in this paper because of its 
importance and to facilitate the presentation of this first 
application of info-gap theory to public health. 

We use info-gap theory [8] to model and manage uncer- 
tainties in the following parameters: slow and fast relapse 
rates for HIV positives and negatives, TB infection rates 
for HIV positives and negatives, and the HIV infection 
rate. Much literature suggests these parameters for their 
impact on the course of epidemics and the difficulty 
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in measuring them [10,11,16,33-36]. Other uncertain- 
ties could also be investigated, depending on the pur- 
pose of the analysis. We use estimated values for each 
uncertain parameter, and estimated errors typically cho- 
sen as half of an interval estimate of the parameter. The 
info-gap model of uncertainty is specified in Appendix 
"Uncertainty" section. 

We aim to achieve the performance requirement by 
judicious choice of control variables, defined in Appendix 
"Control variables" section. Eligible control variables are 
any coefficients of the dynamic model that can be influ- 
enced by public health or related medical intervention. 
We use the diagnosis rate, cure rate, relapse rate, and HIV 
infection rate. We define an intervention in terms of the 
values of these variables [15,34,37-40]. 

Results: robustness and policy evaluation 

We use the info-gap robustness function to evaluate 
alternative interventions aimed at controlling the rela- 
tive TB prevalence, C{t), at a specified target time, tm, 
in the future. An intervention is specified by the values 
of the control variables. The evaluation leads to realis- 
tic assessment of outcomes and preferences among the 
interventions. 

Interpreting robustness curves: trade off and zeroing 

All info-gap robustness curves have two properties, 
mentioned earlier: trade ofi^ between performance and 
robustness, and zeroing of the robustness curve. These 
properties are central in using robustness curves to evalu- 
ate public health policy. 

The coefficients of the epidemiological models are spec- 
ified in Tables 1 and 2. Thoughout our examples, the initial 
conditions correspond to low TB and low HIV prevalence 
(the first data-column of Table 3) unless specified other- 
wise. The control variables specified in Appendix "Control 
variables" section are themselves model parameters. The 
robustness curve in Figure 1 is evaluated for the nominal 
values of the control variables specified in Tables 1 and 2. 
This set of control variables is the "baseline intervention". 
The uncertain variables specified in Appendix "Uncer- 
tainty" section are also model parameters. Their nominal 
values and uncertainty estimates are specified in Table 4. 
These nominal values are the same as appear in Tables 1 
and 2 for these variables. The total case load is evaluated 
at time tm = 10 years after initiation unless indicated 
otherwise. 

Figures 2 and 3 show the temporal evolution of the rel- 
ative prevalence of TB cases, C{t), and relative relapses, 
R{t), based on the nominal estimates of the model param- 
eters, with moderately low initial TB and HIV prevalence. 
C{t) and R{t) are fractions of the initial total population 
size. Figure 2 shows that the total number of TB cases 



starts at about 4.2% of the initial population and decays 
to about 3% in the first 1.5 years, thereafter decaying 
more slowly, reaching 2.1% of the initial population size 
after 10 years. The relapse population starts very small, 
rises rapidly in the first year and thereafter decays grad- 
ually. The reduction in the rate of decrease of the TB 
cases after 1.5 years. Figure 2, results from the influx 
of relapses which have built up since initiation of the 
intervention. 

Trade off 

Key to understanding the trade off expressed by the 

robustness curve is the concept of satisficing. In contrast 
to optimizing, satisficing asks for an outcome that meets 
minimal needs but may not be the best imaginable. The 
satisficing strategy is not merely "accepting second best." 
Satisficing is aspirational, setting a goal just like opti- 
mization, but also requiring robustness to uncertainty. 
The satisficing strategy induces a trade off^ between the 
aspiration for good outcome and the robustness against 
uncertainty in attaining that outcome. 

The robustness curve in Figure 1 is based on satisficing 
the relative TB prevalence: requiring that the prevalence 
not exceed the critical value. Cm- Figure 1 shows the 
robustness vs. the critical prevalence. The positive slope 
of the robustness curve in Figure 1 expresses the trade 
off between robustness and performance: large robust- 
ness entails large prevalence at the specified target time 
(10 years). Equivalently, requiring low relative prevalence 
entails low robustness to uncertainty in the epidemiolog- 
ical model. The robustness curve quantifies the intuition 
that more demanding outcomes (small prevalence) are 
more vulnerable to model uncertainty (small robustness). 

We can interpret the numerical values along the robust- 
ness curve as follows. The prevalence, C{t), and its critical 
value, Cm, are normalized to the initial population size. 
For instance, Cm = 0.025 means that the prevalence at 
time tm must not exceed 2.5% of the initial population 
size. The robustness corresponding to this value of Cm, is 
0.1 as seen in Figure 1. This means that the performance 
requirement is guaranteed if the uncertain model param- 
eters vary from their nominal values by no more than 
10% of their error estimates. (The model parameters are 
constrained to be positive since they are first-order rate 
constants.) 

The public health practitioner may feel that robust- 
ness to 10% uncertainty in the model parameters is rather 
small, given the substantial uncertainty in the epidemio- 
logical dynamics of TB with HIV prevalence. If we want 
robustness to, say, 25% uncertainty in the model parame- 
ters we must accept a larger final case load, namely, Cm = 
0.033 as seen in Figure 1. Greater robustness is obtained 
only by accepting poorer outcome; this is an irrevocable 
trade off that is quantified by the robustness curve. 
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Table 1 Model parameters In the Murray-Salomon basic model 



Symbol 


Definition 


HIV neg 


HIV pos 


Birth rate ^ 


births/year/person 


0.03^ 




N 


population size 


0.821' 


0.179' 


T 


births per year = birth ratexN^ 








infection rate 


1.81 X 10-3 


2.96 X 10-3™ 




# respiratory contacts with infected/person/year 








probability that respir, contact with infectious source leads to infection 






KL 




5-15" 






# infectious cases in population 








non-TB death rate 


0.009<^ 


0.05'^ 


p 


proportion of new infections entering slow breakdown 


0.9 (0.85-0.95)° 


0.4 (0.3-0.5°) 




fast breakdown rate 


2" 


3 


ft 9 


slow breakdown rate 


0,001 C5-15 X lO"'*") 


0.075(0.05-0.10") 




rate of application of INH to infected individuals 


0.75 (*) 




V 


protection from superinfection conferred by primary infection 


0.75(0.5-1") 




w 


short-term INH effectiveness 


0.7 




h 


long-term INH effectiveness 


0.7 






proportion of pre-diagnosed cases in clinical category / entering diagnosis category j 






0(1,1 




0.45 (0.4-0.5y 


0.35 (0.3-0.4/, <^ 






0.55 (0.5-0.6) 


0.65 (0.6-0.7) ^ 








e 






rf,2 = 1 - rf-l 


f 


s' 


proportion of new cases in clinical category / 








proportion of new cases in clinical category / 


0.45 (0.4-0.5") 




s3 


proportion of new cases in clinical category / 






S)<3 


diagnosis rate for category j 


0.6' 


0.6 « 


0-3 


smear conversion rate 


0.03" 





Model parameters in the Murray-Salomon basic model, Table Two, p,42, in ref. [18], except for /(, L, / and N which are defined in footnote 2 on p.21 of [1 8]. Where a 

value is specified only for HIV-negative, the same value is used for HIV-positive. 

^Footnote 8, p.24, [18]. 

"Footnote e. Table A5, p.63, [18]. 

':TableA5,p.63, [18]. 

''Footnote c, Table A5, p.63, [18]. 

''Footnote Table A5, p.63, and Figure A5, p.57 [18]. 

'Footnote b. Table A5, p.63,[1 8]. 

SRate: per person per year. In Botswana the average is 477 cases per year per 100,000 people. 62% of them are HIV infected. 
''Depends on HIV prevalence. In areas with HIV and without preventive treatment, 25% of babies born from HIV mothers are infected. 
'In Botswana. 

j[Dye C, Scheele S, Dolin P, Pathania V, Raviglione MC. Consensus statement. Global burden of tuberculosis: estimated incidence, prevalence, and mortality by country. 
WHO Global Surveillance and Monitoring Project. J/\MA. 1999 Aug 1 8;282(7):677-86,]. 

^Not needed since A is a primary datum. 
'Decision variable. 

""Can also be treated as a decision variable. 



Zeroing 

We note that the robustness curve in Figure 1 reaches the 
horizontal axis at the value Cm = 0.021. This means that 
requiring the prevalence not to exceed 2.1% of the initial 
population has no robustness against model uncertainty. 
The value of Cm at which the robustness becomes zero 



is precisely the nominal prediction of the prevalence at 
time tm as seen by the right end-point in Figure 2. That 
is, the value of C(tm): evaluated with the best estimates of 
the model parameters, equals 0.021. The horizontal inter- 
cept in Figure 1 is an example of the property of zeroing 
that holds for all info-gap robustness curves: The outcome 



Ben-Haim etal. BMC Public Health 2012, 12:1091 
http://www.biomedcentral.eom/1 471-2458/1 2/1 091 



Page 6 of 17 



Table 2 Model parameters In the Murray-Salomon basic model 



Symbol 


Definition 


HIV neg 


HIV pes 




spontaneous cure rate for untreated cases 


0,2(0.14-0.25)9 






TB death rate for untreated cases in clinical category / 










0.12(0.075-0.20") 


045 (0.3-0.6'') 


Mu. 1^1 




0.7 tx^^ 




gl.k 


proportion of treated cases in clinical category / and treatment category /< 










05'' 




g2,l 




0.28'' 




g3,l 




d 




g/.2 




ga = 1 _ g'.l 




c,k f 
Sj 


cure rate for treated case in treatment category k 






4 




0.8 








0.5^ 




i^'f 


TB death rate for treated cases in clinical category / and treatment category k 






i^Y 




0.075"^ 


0.16^ 


i^Y 




0.12" 


0.24*^ 






0.7/x|'* 


0.7^1}'" " 




proportion of spontaneously recovered cases entering the slow relapse category 


0.009^ 






proportion of recovered cases from treatment category k entering the slow relapse category 






4 




0.0096"^ 








0.0094"^ 






fast relapse rate 


2^ 


3 




slow relapse rate 


0.001 (5-15 X lO-"*") 




y' 


rate of HIV infection 


0.075(0.011-0.95)'' 





Model parameters In the IVIurray-Salomon basic model, Table Two, p.42, in ref. [1 8], except for K, L, Y and N wliich are defined in footnote 2 on p.21 of [1 8], Wliere a 
value is specified only for HIV-negative, the same value is used for HIV-positive. 
=Footnote8,p.24,[18]. 
''Footnote e. Table AS, p.63, [1 8]. 
■^TableAS, p.63, [18]. 

"•Footnote t. Table A5, p.63, and Figure A4, p.56 [1 8]. 
^Footnote d. Table A5, p.63, [1 8]. 
'Rate: per person per year. 
9[33,34]. Other estimate: 0.05-0.1 5". 

predicted by the model, when adopted as the performance 
requirement, has no robustness against uncertainty in 
the model. 

It is not surprising that the predicted outcome is 
extremely vulnerable to error in the model upon which the 

prediction is based. However, the zero-robustness of pre- 
dicted outcomes has an important implication for policy 
selection. 

The robustness curve in Figure 1 is for a particular 
choice of values of the control variables: the baseline 
intervention. The zeroing property— no robustness of the 
predicted outcome of these control values— implies that 
we should not assess these control values in terms of 
their predicted outcome. The predicted prevalence of 
0.021 at time tm = 10 years does not reliably reflect 



the performance of these control variables. Due to the 
trade off property, only larger prevalence can reliably be 
expected to result from this choice of the control vari- 
ables. Predicted outcomes are not reliable for prioritizing 
the interventions. 



Equivalent interventions 

Different combinations of interventions can yield essen- 
tially equivalent results, as in Figure 4. The baseline 
intervention (solid), is characterized by low diagnosis rate 
and high relapse rate. The other intervention (dash) has 
higher diagnosis rate and lower relapse rate as specified in 
Table 5. (Interventions are specified by the values of con- 
trol variables presented in Table 5). The robustness curves 
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Table 3 Initial conditions 





Symbol 




Low TB Prevalence 






Medium TB Prev. 






HighTB Prev. 








Low 


Med 


High 


Low 


Med 


High 


Low 


Med 


High 






HIV 


HIV 


HIV 


HIV 


HIV 


HIV 


HIV 


HIV 


HIV 


1 


U 


0.9 


0.8 


0.6 


0.8 


0.650 


0.500 


0.6 


0.5 


04 


2 


If 


0.0075 


0.015 


0.03 


0.011 


0.018 


0.028 


0.03 


0.0375 


0.045 


3 


Is 


0.03 


0.06 


0.12 


0.05 


0.088 


0.125 


0.12 


0.15 


0.18 


4 


Sf 


0.003 


0.006 


0.012 


0.01 


0.018 


0.025 


0.012 


0.015 


0.018 


5 


Hs 


0.009 


0.018 


0.036 


0.02 


0.035 


0.050 


0.036 


0.045 


0.054 




6 


('J) = (1,1) 


0.005 


0.01 


0.02 


0.008 


0.014 


0.020 


0.02 


0.025 


0.03 


7 


Oj) = (2,1) 


0.002 


0.004 


0.008 


0.003 


0.005 


0.008 


0.008 


0.01 


0.012 


8 


('J) = (3,1) 


0.002 


0.004 


0.008 


0.003 


0.005 


0.008 


0.008 


0.01 


0.012 


9 


('J) = (1,2) 


0.002 


0.004 


0.008 


0.003 


0.005 


0.008 


0.008 


0.01 


0.012 


10 


(/J) = (2,2) 


0.001 


0.002 


0.004 


0.002 


0.004 


0.005 


0.004 


0.005 


0.006 


11 


(/J) = (3,2) 


0.001 


0.002 


0.004 


0.002 


0.004 


0.005 


0.004 


0.005 


0.006 




12 


(a) = (1,1) 


0.01 


0.02 


0.04 


0.02 


0.035 


0.050 


0.04 


0.05 


0.06 


13 


(a) = (2,1) 


0.005 


0.01 


0.02 


0.01 


0.018 


0.025 


0.02 


0.025 


0.03 


14 


(a) = (3,1) 


0.005 


0.01 


0.02 


0.01 


0.018 


0.025 


0.02 


0.025 


0.03 


15 


(a) = (1,2) 


0.005 


0.01 


0.02 


0.02 


0.035 


0.050 


0.02 


0.025 


0.03 


16 


(a) = (2,2) 


0.002 


0.004 


0.008 


0.005 


0.009 


0.013 


0.008 


0.01 


0.012 


17 


(a) = (3,2) 


0.0025 


0.005 


0.01 


0.003 


0.005 


0.008 


0.01 


0.0125 


0.015 


18 


Rf 


0.002 


0.004 


0.008 


0.005 


0.009 


0.013 


0.008 


0.01 


0.012 


19 


fts 


0.006 


0.012 


0.024 


0.015 


0.025 


0.038 


0.024 


0.03 


0.036 






1.000 


1.000 


1.000 


1.000 


1.000 


1.000 


1.000 


1.000 


1.000 




0.02 0.025 0.03 0.035 0.04 



Critical prevalence 

Figure 1 Robustness of relative TB prevalence. Run 8 



for these two control strategies, at 10 years, are nearly the 
same, suggesting that the public health practitioner may 
choose freely between them, perhaps employing addi- 
tional criteria such as cost or ease of implementation. 
Equivalence may be lost if parameters are changed. For 
instance, we will see later (Figure 5) that these interven- 
tions evaluated at 10, 20 or 30 years have very different 
robustness curves. 



Table 4 Nominal values and error weights of uncertain 
variables 



Symbol 


Nominal value, u, 


Error weight, s; 


A, 


1.81 X 10-3 


0.0009 


X 


2.96 X 1 Q-'^ 


0.0018 


V 


0.075 


0.2 


PS 


0.001 


0.0005 


Ps 


0.001 


0.001 


PF 


2 


1 



PF 3 1.5 
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0.045 




Time (yr) 

Figure 2 Relative TB prevalence vs. time. Run 8. 



Figure 6 shows a different aspect of the equivalence 
of interventions. The figure shows robustness curves for 
two strategies specified in Table 5. Both strategies aim to 
control the relative prevalence of TB, but one (solid) is 
geared for a 10-year target time, while the other (dash) 
considers a 30-year target. The estimated outcomes — 
prevalence — are very nearly the same for these two 
strategies, each at its respective target time, as shown by 
their shared horizontal intercept at Cm = 0.018. These 
predictions result from estimated model parameters, so 
one might be inclined to conclude that TB prevalence of 
0.018 can be achieved at either 10 or 30 years by using the 
corresponding intervention. 




Time (yr) 

Figure 3 Relative relapses vs. time. Run 8. 




0.02 0.025 0.03 0.035 0.04 

Critical Prevalence 

Figure 4 Equivalent robustness for two interventions. Run 8: — , 

run 15:--. 



However, the epidemiological model is highly uncertain, 
and the robustness curves in Figure 6 of these two strate- 
gies are quite different. Not surprisingly, the 30-year target 
is much less robust to uncertainty. It would be erroneous 
to treat these two strategies as outcome-equivalent since 
their performances at positive robustness are quite dif- 
ferent. Nominal equivalence (equivalence of the predicted 
outcome) does not imply robustness equivalence. 

Impact of initial TB and HIV prevalence 

We now consider higher initial prevalences. The overall 
shape of the dynamic response is very similar in each case, 
except that the prevalence increases significantly as the 
initial prevalence increases. As in Figures 2 and 3, in each 
scenario the initial TB prevalence decreases rapidly dur- 
ing the first 2 years, and thereafter decreases more slowly 
as the new relapse population — which peaks around the 
end of the first year — flows back into active cases. 

Figure 7 shows robustness curves for a target time 10 
years after initiation, for low (solid), medium (dash) and 
high (dot-dash) initial prevalence of TB and HIV. The 
low-prevalence curve (solid) is the same as Figure 1. The 
robustness curves shift dramatically to the right as the 
baseline prevalence of TB and HIV increases, indicat- 
ing poorer estimated outcome and lower robustness to 
uncertainty. 

Intervention aggressiveness 

Figure 8 shows robustness curves for low initial TB and 
HIV prevalence with interventions specified in Table 5. 
The solid curve is the baseline intervention, against which 
the other curves entail more aggressive intervention in 
either or both the active cases and the relapse population. 
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Table 5 Control variables for robustness curves 



Run 


Init 
Prev' 


tm 

(yr) 


&i = s' 


®r 


= 4 


PF 


Pf 


A, 


X 


Y 


Pf 


Pf 


8 




10 


(0.6, 0.6) 


(0.£ 


, 0.5) 


2 


3 


0.00181 


0.00296 


0.075 


2 


3 


9 




10 


(0.65, 0.65) 


(O.f 


, 0.5) 


2 


3 


0.00181 


0.00296 


0.075 


2 


3 


10 




10 


(0.65, 0.65) 


(O.f 


8, 0.55) 


2 


3 


0.00181 


0.00296 


0.075 


2 


3 


11 




10 


(0.65, 0.65) 


m 


8, 0.55) 


1.5 


2.25 


0.00181 


0.00296 


0.075 


2 


3 


12 




10 


(0.65, 0.65) 


(0.E 


8, 0.55) 


1 


1.5 


0.00181 


0.00296 


0.075 


2 


3 


15 




10 


(0.85, 0.85) 


(0.£ 


, 0.5) 


1.2 


2 


0.00181 


0.00296 


0.075 


2 


3 


19 


5 


10 


(0.6, 0.6) 


(O.f 


, 0.5) 


2 


3 


0.00181 


0.00296 


0.075 


2 


3 


21 


5 


10 


(0.65, 0.65) 


(O.f 


, 0.5) 


2 


3 


0.00181 


0.00296 


0.075 


2 


3 


22 


5 


10 


(0.65, 0.65) 


(O.f 


8, 0.55) 


2 


3 


0.00181 


0.00296 


0.075 


2 


3 


23 


5 


10 


(0.65, 0.65) 


(0.£ 


8, 0.55) 


1 


1.5 


0.00181 


0.00296 


0.075 


2 


3 


20 


9 


10 


(0.6, 0.6) 


(0.f 


, 0.5) 


2 


3 


0.00181 


0.00296 


0.075 


2 


3 


24 


9 


10 


(0.65, 0.65) 


(0.f 


, 0.5) 


2 


3 


0.00181 


0.00296 


0.075 


2 


3 


25 


9 


10 


(0.65, 0.65) 


(O.f 


8, 0.55) 


2 


3 


0.00181 


0.00296 


0.075 


2 


3 


26 


9 


10 


(0.65, 0.65) 


(O.f 


8, 0.55) 


1 


1.5 


0.00181 


0.00296 


0.075 


2 


3 


27 




20 


(0.6, 0.6) 


(0.f 


, 0.5) 


2 


3 


0.00181 


0.00296 


0.075 


2 


3 


28 




30 


(0.6, 0.6) 


(0.f 


, 0.5) 


2 


3 


0.00181 


0.00296 


0.075 


2 


3 


29 




10 


(0.6, 0.6) 


(0.E 


, 0.5) 


2 


3 


0.00181 


0.00296 


0.0375 


2 


3 


30 




10 


(0.6, 0.6) 


(O.f 


, 0.5) 


2 


3 


0.00181 


0.00296 


0.05 


2 


3 


31 




10 


(0.6, 0.6) 


(O.f 


, 0.5) 


2 


3 


0.00181 


0.00296 


0.06 


2 


3 


32 




10 


(0.6, 0.6) 


(O.f 


, 0.5) 


2 


3 


0.0009 


0.00148 


0.075 


2 


3 


33 




10 


(0.6, 0.6) 


(0.£ 


, 0.5) 


2 


3 


0.0003 


0.0005 


0.075 


2 


3 


38 




30 


(0.85, 0.85) 


(0.E 


, 0.5) 


2 


3 


0.00181 


0.00296 


0.075 


2 


3 


39 




10 


(0.6, 0.6) 


(O.f 


, 0.5) 


2 


3 


0.00181 


0.00296 


0.075 


1 


1.5 


41 




10 


(0.65, 0.65) 


(0.f 


, 0.5) 


2 


3 


0.00181 


0.00296 


0.075 


1 


1.5 



Data-column in Table 3. 



The progression from solid to dashi to dot-dash in 
Figure 8 represents increasingly aggressive intervention 
in the active TB case population. We see that increasing 
aggressiveness, in this specific parameter configuration, 
results in increasing prevalence and decreasing robust- 
ness to model error at the target time. The explanation 
is that aggressive treatment of active cases enlarges the 
relapse population which flows back into the active case 
population. 

The top curve in Figure 8 modifies the most aggressive 
case (dot-dash) by also including more aggressive inter- 
vention in the TB relapse population. This reduction in 
relapse reduces the predicted prevalence after 10 years, 
and increases the robustness to uncertainty. 

Different target times 

Most of the results discussed so far evaluated the robust- 
ness for a target time 10 years after initiation. We now 
consider the implications of different target times. 



Fi gure 5 shows robustness curves at target times, 
of 10, 20 and 30 years (solid, dash, dot-dash respec- 
tively). The initial prevalences of TB and HIV are low. The 
interventions are all at the baseline. 

The predicted prevalence decreases as the target time 
increases, as shown by the horizontal intercepts in 
Figure 5. The baseline intervention is predicted to reduce 
the prevalence, (in units of initial population size), as 
the time horizon increases. However, the zeroing prop- 
erty means that these predictions have no robustness to 
uncertainty in the model used for prediction. Only higher 
prevalence has positive robustness. 

From Figure 5 we see that, for critical TB prevalence Cm 
less than 3%, the 30-year TB prevalence is more robust 
than the 20-year prevalence which is more robust than 
the 10 year prevalence. For instance, at critical TB preva- 
lence of Cm = 0.02, the robustnesses for 10, 20 and 30 
year horizons are 0, 0.08 and 0.12, respectively. This inter- 
vention has no robustness to uncertainty when requiring a 
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'30 yr 



0.02 0.04 0.06 

Critical prevalence 

Figure 5 Robustness curves at 1 0, 20 and 30 years. Run 8: — , run 

27:--, run 28:-. 




0 .05 0.1 .15 0.2 

Critical prevalence 

Figure 7 Robustness curves for low, medium and high initial TB 
and HIV prevalence. Run 8: — , run 1 9: - - run 20: — . 



2% prevalence after 10 years; in fact, the estimated preva- 
lence at 10 years is greater than 2%. The prevalence at 20 
years will be no worse than 2% provided that the model 
coefficients err by no more than 8%, and at 30 years the 
robustness to error is 12%. 

The practitioner may feel that even 12% robustness 
against model-coefficient error is rather small, given the 
severe uncertainty of TB epidemiology in the context of 
epidemic HIV. This means that, even at a 30-year hori- 
zon, this intervention cannot reliably achieve a relative 
prevalence as low as 2%. 



Suppose we are willing to aim at a final TB prevalence of 
3.7%. We see from Figure 5 that now the 10-year horizon 
is more robust than 20 years which is more robust than 30 
years. The robustnesses are now 30%, 24% and 22% for 10, 
20 and 30 years. The robustness curves have intersected 
one another and the robustness rankings are reversed. As 
the target time decreases, the predicted outcome becomes 
worse (horizontal intercept moves right) but the cost of 
robustness improves. This causes the robustness curves to 
cross one another. More intuitively, we can say that pre- 
diction of TB prevalence is more reliable for short time 



0.25 
S 0.2 
'I 0.15 
O 0.1 

0.05 
0 





■ 1 ■ 4 

/10yr 




/ >> 

/ ^ 

1 ■ ^ ■ 

1 * 




\ /30 yr 

/ /. , 




/ 

: / : 
: / : 


/ / 


: / 
/ 


fl 





0 0.025 0.05 0.075 0.1 

Critical prevalence 

Figure 6 Nominal equivalence of two interventions. Run 1 2: — , 

run 38: - -. 




0.02 0.03 0.04 

Critical prevalence 



Figure 8 Robustness with varying aggressiveness. Run 8: — , run 

9:--, run 10:—, run 12: • • •. 
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horizon than for long times. But since a long time is 
required to overcome the relapse effect, we observe the 
intersection of the robustness curves and the consequent 
reversal of their robust dominance. 

Results like Figure 5 have important policy implications 
for TB control over long time periods. The policy maker 
may be tempted to choose one option that is predicted to 
yield better short term results. However, that choice might 
be wrong when one opts to satisfice the outcome with 
robustness to uncertainty. Predictions of mathematical 
models (horizontal intercepts) are not sufficiently reliable 
for comparing and prioritizing interventions; the cost of 
robustness (slope) must also be considered. In the exam- 
ple in Figure 5 one might conclude that prevalence less 
than 3% is not achievable at any target time, that 3.7% 
is feasible at 10-years but not beyond, and that other 
interventions are needed for longer-term outcomes. 

Impact of HIV mortality 

Figure 9 shows 10-year robustness curves for various HIV 
infection rates, with low initial TB and HIV prevalence, 
as specified in Table 5. The HIV infection rate decreases 
in the progression from solid, dash, dot-dash to dot-dot. 
As the HIV infection rate decreases, the estimated 10-year 
TB prevalence increases and the robustness decreases. 
The explanation lies in the high mortality rate of the 
HIV population. As the HIV infection rate decreases, the 
size of the relapse population decays more slowly, allow- 
ing greater flow back into the active TB case population. 
Interventions that decrease HIV infection rates or restore 
immunity to HIV patients, will counter-intuitively tend to 




0.02 .025 0.03 .035 0.04 

Critical prevalence 

Figure 9 Robustness for various HIV infection rates. Run 8: — , run 

31:--, run 30: -, run 29: ■ ■ ■. 



increase TB prevalence unless compensating measures are 
taken. Significantly, the cost of robustness (slope of the 
robustness curve) does not change as a result of decreased 
HIV infection rate. Reducing HIV infection rate shifts 
the robustness curve to the right, with almost no change 
in slope. 



Conclusion 

We demonstrated a generic info-gap framework for 
managing model uncertainty in public health decision 
making. By applying it to a mathematical model of 
TB/HIV epidemics, we illustrated specific recommenda- 
tions for interventions in the control of TB with HIV in 
various settings. 

The complicated multi-dimensional epidemiological 
dynamics are dominated by the flow back and forth 
between the actively and latently infected TB populations 
and the different rates of progression of different subpopu- 
lations between these compartments. Counter-intuitively, 
the total TB case load even decades after initiation can 
increase as a result of increased diagnosis and cure rates, 
and it can increase as the control of HIV becomes more 
aggressive. These findings highlight the critical impor- 
tance of modeling in the assessment and planning of 
public health intervention. Model predictions are often 
used to choose interventions. However, model predic- 
tions must be interpreted in light of model uncertain- 
ties. Predicted outcomes have zero robustness to model 
error. Only worse-than-predicted outcomes (higher rel- 
ative prevalence) have positive robustness against model 
error This means that predicted outcomes are not reliable 
for prioritizing the interventions. The trade off between 
robustness and outcome is quantified by the info-gap 
model analysis and is a critical component of the decision- 
making process. 

We explore the performance of interventions that alter 
the rate constants of diagnosis, cure, relapse and HIV 
infection. Some interventions have quite similar predicted 
outcomes and robustness curves. This enables the policy 
maker to choose between these interventions based on 
additional criteria, such as ease or cost of implementation. 
It is not true, however, that interventions with the same 
estimated outcomes necessarily have the same robustness 
against model error. 

We demonstrate the policy implications of initial TB 
and HIV prevalence, of HIV mortality, of degree of treat- 
ment aggressiveness, and of the target time at which 
outcomes are evaluated. Public health policies are evalu- 
ated in terms of confidence — expressed as robustness to 
modeling error — in achieving specified TB prevalence at 
the target time. Predicted outcomes have zero robustness 
and thus are not reliable for evaluating and comparing 
interventions. Instead, interventions must be prioritized 
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in terms of their capacity for achieving specified out- The basic Murray-Salomon model: No HIV 
comes, with robustness to uncertainty. Failure to quantify The basic M-S model is the following 19 differential 
the uncertainty inherent in public health interventions equations (eqs.(6) and (7) occur in 6 different forms each) 
leads to disappointment from unrealized expectations, appearingonpp.l9-20of Murray and Salomon [18]: 
and failed policy. Where a public health model under- 
lies guidelines, info-gap decision theory provides valu- 
able insight into the confidence of achieving agreed-upon 
goals. 

Appendices 

The Murray-Salomon model 

The Murray-Salomon (M-S) model [17,18] is a set of 
coupled differential equations that describe the time evo- 
lution of TB. A modification deals with TB infecteds in a 
population containing HIV smear-positive individuals. In 
section "The basic Murray-Salomon model: No HIV" we 
define the basic non-HIV model. In section "The HIV-Ex- 
tended model" we present the M-S extension to include 
an HIV sub-population. The state variables are defined in 
Table 6 and the parameters are defined in Table 1. 



Table 6 State variables — sizes of sub-populations — in the IMurray-Salomon basic model. Table One, p.41 , in ref . [1 8] 



index 


Symbol 


Definition 




initial value' 










HiV Neg 


HiV pos 


Ref. 


1 


U 


Uninfected 


0.9'' 


0.2": 


[10] 


2 


If 


Infected subject to fast breakdown 


0,05-0.1 


0.1 


[33,34] 


3 


Is 


Infected subject to slow breakdown 


0,1 


0.1 


[33,34]] 


4 


Sf 


Superinfected subject to fast breakdown 


0 






5 


Hs 


INH recipient subject to slow breakdown 


0,01 


0.01-0,03 




6-11 


Hi 


Untreated cases, of 5 types: 


0 






/ = 1: smear-positive pulmonary 


/ = 2: smear-negative pulmonary 


/ = 3; extra-pulmonary 


j = V. fast diagnosis category 


= 2: slow diagnosis category 


/■* = 2if/ = 1 (eqs,(6)and (7)) 


/■* = 1 if; = 2(eqs,(6)and (7)) 


/■* = 0if/=3(eqs.(6)and (7)) 


12-17 


Cf 


Treated cases, of 6 types: 


0 






/ as above 


k= ^ ■. good treatment category 


k=2: bad treatment category 


18 


Rf 


Recovered cases subject to fast relapse 


0,4 (0.28-0.52) 




[33,34] 


19 


Rs 


Recovered cases subject to slow relapse 


0,050 (0.035-0.065) 




[33,34] 



The definition of superscript is in [1 8] on p.20. 
"As fraction of total population at start of simulation. 
''Low prevalence. 
'^Higfi prevalence. 



-- = T-XU-tiU (1) 
dr 

dip 

= (1 - p)XU - fiplF - wxIf - I^If (2) 

at 
d/c 

-f=pXU- fish - Xh - (1 -/')A(1 - v)Is - Ills (3) 
at 



^ = il-p)X(l-v)iIs+Hs+Rs)-fiFSF-wxSF-l^SF 
at 

(4) 

^ = X (wIf + wSf + Is) - (1 - P)Hi - v)Hs 
at (5) 

- (1 - h)fisHs - ixHs 
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dt 



" = {MIf + Sf) + Mis + (1 - h)Hs] 
+PsRs + PfRf] d'''s' 

-S'C^±aC'^/ -suC'^-(li + li'u)c'ii, (6) 
for i = 1, 2, 3 and J = 1,2 



gi.k J2 S^C^ ± ad;-' - e\Cf - (/X + M'f')4^ (7) 



dt 



dRp 
~dF 



j=l,2 



for (■ =1,2,3 and k = 1,2 



1=1,2,3 

i=1.2 



- pfRf - iJiRp 



dRs^ 
dt 



k=\,2 



(=1,2,3 



(8) 



1=1,2,3 
/•=1,2 



1=1,2,3 

PsRs - (1 -p)X{\ - v)Rs - txRs 



(9) 



The term '±a' appears in eqs.(6) and (7). M-S write:^ 

It should be noted in the equations for Cy and 
that the smear rate a is multiphed by the number of 
individuals in the respective category i*, where i* = 2 
(smear-negative) for / = 1 (smear-positive) and vice 
versa, and i* — 0 for / = 3 (extra-pulmonary). The 
term including a is added for i = 1, subtracted for 
i = 2, and equal to 0 for / = 3. The result of this 
formulation is that smear-negative patients convert 
to smear-positive at a rate of cr. 

However, the 'vice versa' is a mistake. The correct 
equations for C^ (with analogs for Cj^) are: 



dC 



dt 



= ... + aCy + ■■■ 



dC 



2,j 



dt 



dC 



3,j 



dt 



= ■■■ + OC'u'' + ■■■ 



(10) 



(11) 



(12) 



Eq.(lO) states that smear-negative individuals join the 
smear-positive population at rate a. Eq.(ll) states that 
smear-negative individuals leave the smear negative pop- 
ulation at rate a. That way all individuals are accounted 
for. 



The instantaneous rate of infection, X in eq.(l), is 
defined by Murray and Salomon [18], p. 21, as: 



■ KL- 



N 



(13) 



The HIV-Extended model 

Introduction 

We will now formulate the extended dynamic model to 
include a differentiation between HIV-positive and HIV- 
negative populations. M-S do this also, and state [18], p.4 
that they use "two sub-models — one for the HIV sero- 
negative population, and one for the HIV sero-positive 
population. Each sub-model follows the structure" which 
is presented here as eqs.(l)-(9). They write that 

Individuals move from each category in the 
HIV-negative sub-model to the corresponding 
category in the HIV-positive sub-model at the HIV 
infection rate, which varies over time. Because the 
effects of HIV on immune function are not marked 
with respect to tuberculosis until the CD4 count has 
dropped below 500, we actually move individuals 
from the HIV-negative to the HIV-positive 
sub-model after they have been infected with HIV for 
3 years. The two sub-models are also linked through 
the annual risk of infection, as HIV-negative 
tuberculosis cases can infect HIV-positive 
individuals, and vice versa [18], pp.4-5 

Our model does not delay transfer from the HIV- 
negative sub-model. 

Sub-models Each of the two sub-populations — HIV- 
negative and HIV-positive— is divided into the 19 groups 
represented by the state variables in Table 6. Each state 
variable has a differential equation in eqs.(l)-(9). 

Let us denote the HIV-negative state variables as before, 
and the HIV-positive state variables with the same letters 
but with an over-bar. For compactness we represent these 
two sets of variables with two vectors: 

^ I J f T T c T-f /^Ijl /^2,1 /--'3,1 /--'1,2 f-^2,2 /--'3,2 

X — I '-^ ' '■Fi ^St '^Ft ^St ' ' U * U ' U ' U ' 

, ,Cj> iCj- >-^F'-^s^ (1^) 

- (ttI 1 c Tj T^l'l 7^2,1 r^l,2 7^2,2 y,i,2 

X — [U,ip,is, Of iK^jj jLy t'^u I'^u ' 

C^T,cf,cf, cf, cf,RF, Rs) (15) 

The model parameters Usted in Tables 1 and 2 take 
different values for HIV-negative and HIV-positive pop- 
ulations (as specified in the tables). Let us denote the 
model parameters as before for the HIV-negative popula- 
tion, and use the same symbols with an over-bar for the 
HIV-positive population. 
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Eqs.(l)-(9) are Ist-order linear inhomogeneous differ- 
ential equations. Only eq.(l) has an inhomogeneous term: 
T births per year. Let F{t) and F(t) denote the matri- 
ces of coefficients (model parameters) in the differential 
equations for HIV-negative and HIV-positive populations, 
respectively. Let ei denote the 19-vector with a 1 in the 
first element and zeros elsewhere. We can now compactly 
denote eqs.(l) as: 



dx 
di 



■■F(t)x + eiT 



(16) 



Let / denote the HIV infection rate, per person per 
year. Following M-S, we will move individuals from each 
HIV-negative category to the corresponding HIV-positive 
category at rate y. Thus, instead of eq.(16), we have the 
following coupled sets of equations: 



dx 
di 



= F(t)x + eiT — yx 



dx 



= F(t)x + eiT + yx 



(17) 



(18) 



The term '—yx' in eq.(17) removes individuals from the 
HIV-negative population at the HIV infection rate, and 
the term '+yx' in eq.(18) introduces them into the HIV- 
positive population at the same rate. 

M-S introduce further highly structured coupling 
between eqs.(17) and (18) through the TB infection rate, 
[18], p.23, A. We do not employ the M-S differentiation 
between the infection rates for HIV-negative and HIV- 
positive populations. Instead we simply use A and k for the 
TB infection rates in the HIV-negative and HIV-positive 
populations. 

Uncertainty 

Many uncertainties accompany the dynamic model. We 
concentrate on uncertainty in the values of some of the 
model parameters, as this is the dominant impact of HIV 
prevalence. We use info-gap theory to model and manage 
these uncertainties [8]. Many different types of info-gap 
models of uncertainty are available. We employ a model 
particularly suited to severe lack of information. 
The dominant uncertain parameters are: 

PS, Ps> slow relapse rates. 
Pf> Pf' f^st relapse rates. 
X, X, TB infection rates. 
y, HIV infection rate. 

Let us denote uncertain variables generically as w,-, com- 
piled in a vector u. This vector is: 



For each uncertain parameter, m,-, we have an estimated 
value, denoted m„ and an error term Si typically cho- 
sen as half of an interval estimate of the parameter. The 
error estimate may be derived from a statistical confi- 
dence interval, or from a plausible extension of a confi- 
dence interval as discussed by Grassly et al [32], or from 
other professional judgment. The basic idea of an info-gap 
model of uncertainty is that we don't know how wrong our 
estimate is; we have no reliable estimate of a worst case. In 
fact, since the typical values are poorly known, worst-case 
estimates are even less reliable. 

More precisely, the fractional error of the estimate, m,-, 
in units of the error, Si, is unknown. That is, this fractional 
error is bounded by a number, a, whose value is unknown: 



< a, a > 0 



(20) 



But this must be further refined to reflect the fact 
that the uncertain parameters are Ist-order removal-rate 

constants'', which means that they cannot be negative. 
Thus we adjoin these constraints to the inequality as: 



Ui > 0, 



Ui - Ui 



Si 



<a, a > 0 



(21) 



Finally, we write our info-gap model of uncertainty as a 
family of nested sets of uncertain vectors: 



Uia) = \ u : Ui > 0, 



Ui - Ui 



u = (ps, Pf, X, ps, Pf, K y) 



(19) 



< a, for all z| , a > 0 
(22) 



a is called the 'horizon of uncertainty! When a = 0 
there is no uncertainty and the set li{Q) contains only 
the estimated values, u. As a increases, the sets U{a) 
become more inclusive. These sets are unbounded in the 
space on which the parameters are defined. The info-gap 
model embodies the information we have— estimates and 
errors— without committing to any meaningful worst case 
(other than the limits which are imposed by the definition 
of the variables). 

In some situations one may not be able to estimate error 
weights, Si. In such situations the fractional error in eq.(20) 
can be replaced by a fractional error relative to the esti- 
mate, \{ui — Ui) jui I . The info-gap model is then formulated 
as in eq.(22) with this new fractional error. 

Robustness: formulation 
Performance requirements 

We will consider an aggregated variable for monitoring 
the TB status of the population. Our goal is to keep 
the value of this variable acceptably small. The variable 
we consider is the total number of cases, untreated and 
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treated, HIV-positive and HIV-negative, as a fraction of 
the initial population: 

at) = J2 [C'iiit) + C'^it) + C'^jit) + t:i.{t)\ (23) 
Uj 

There are other variables that one could consider. For 
instance, one could consider the total number of relapses, 
fast and slow, HIV-positive and HIV-negative, as a fraction 
of the initial population: 

R{t) = Rpit) + Rsit) + Rfit) + Rs(t) (24) 

One could also consider the instantaneous or the average 
rates of change of C{t) and R{t). 

Returning to the aggregate prevalence, C{t), our goal is 
to keep it below a specified maximum acceptable value at 
a specified target time tm- Thus the performance require- 
ment is: 

Citm) < Cm (25) 

A relation such as eq.(25) is called a "satisficing" require- 
ment, as opposed to an optimization requirement. We do 
not aim to minimize the aggregate prevalence, C(tm)- Our 
goal is to make the TB prevalence adequately small: no 
greater than the critical value Cm, as stated in eq.(25). 
Note that the satisficing requirement includes optimiza- 
tion as a special case. Satisficing and optimizing are the 
same when Cm is chosen as the predicted minimal value. 

Control variables 

We aim to achieve this goal by judicious choice of control 
variables that we denote generically as qi, combined in a 
vector q. Eligible control variables are any coefficients of 
the dynamic model that can be influenced by public health 
or related medical intervention. When a control variable 
is also an info-gap uncertain variable we will refer to the 
estimated value as the control variable. The uncertainty is 
then in whether the specified value— the estimate— will be 
realized in practice. We will consider the following control 
variables: 

S^, I', diagnosis rates (same for HIV negative and 
positive populations). 

e^, e^, cure rates for treateds (same for HIV negative 

andj)ositive populations). 

Pf, Pf- estimated fast relapse rates. 

X, X, estimated TB infection rates. 

y, estimated HIV infection rate. 

Pf> Pf' f^st breakdown rates. 

We define an intervention in terms of the values of these 
variables. None of these control variables corresponds 
directly to any of the standard performance measures such 
as the incidence, prevalence, and death rates associated 
with TB. For instance, the coefficients ^ and S', while 
called "diagnosis rates" are in fact Ist-order kinetic rate 



coefficients and can meaningfully take any positive value. 
These coefficients combine with several other coefficients 
to determine the fraction of new untreated cases that 
move into the treated category, as seen from eqs.(6) and 
(7). In other words, the control variables combine to pro- 
duce aggregate effects such as the proportion of new cases 
that are diagnosed. One can "calibrate" a set of control 
variables in terms of aggregate properties, for instance by 
keeping track of how many cases are created (new mem- 
bers of Cu(t)) and how many are treated (new members 
of Crit)). Unless the population is at steady state (and 
the intervention tries to prevent this), the calibration in 
terms of the proportion diagnosed depends on the time 
after initiation of intervention and on the duration dur- 
ing which the accounting is done. We do not calibrate our 
model since we focus on a different challenging problem: 
prioritizing alternative interventions. 

Definition of robustness 

An intervention is specified by specifying the values of the 
control variables, q. If our dynamic model were accurate 
we could evaluate any proposed intervention in terms of 
the outcome of that intervention that is predicted by the 
model. An intervention whose predicted outcome entails 
low TB prevalence is preferred over an intervention with 
larger predicted prevalence. 

The problem is that the dynamic model is highly uncer- 
tain. This means that it is unrealistic to prioritize inter- 
ventions in terms of their predicted outcomes. Since those 
predictions are highly uncertain, it is unwise to eval- 
uate interventions only in terms of their model-based 
predictions. 

The model-based predictions are useful, but we also 
ask: how wrong could the model be, and the predicted 
outcome is still acceptable? That is, for any specified inter- 
vention, q, we ask: what is the largest fractional error in 
the uncertain parameters, up to which all realizations of 
the model would yield acceptable outcomes? The answer 
to that question is the robustness function, which we will 
soon specify. We use the robustness function to priori- 
tize the interventions in terms of their robustness against 
uncertainty for achieving the required outcomes. 

The robustness function for the performance require- 
ment in eq.(25) is: 

oi(q, Cm) = max I a : ( max C{tm, «) I < Cm [ (26) 

[ \u€U{a) J J 

We can "read" this relation from left to right as follows. 
The robustness, a, of intervention q, with performance 
requirement Cm, is the maximum horizon of uncertainty, 
a, up to which the maximum aggregate prevalence, C{t), 
for all realizations of the uncertain coefficients u in the 
info-gap model U (a), does not exceed the critical value, 
Cm- We are not ameliorating a worst case; the worst 
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case is unknown because the horizon of uncertainty, a, is 
unbounded. Instead, we are asking how large an uncer- 
tainty can be tolerated by the intervention, q. In choosing 
the intervention to enhance the robustness, we attempt to 
protect against the unbounded uncertainty of the impact 
of HIV/ AIDS on the TB dynamics. 

Endnotes 

^Footnote 1 in the full online version, pp. 20-21. 
''This means that these parameters are the coefficients 
in equations such as x(t) = —ux{t) whose solution is 
x{t) = xiO)e^"K In order for this to be a removal process, 
the coefficient u must be positive. It can exceed unity. 
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